Papers with decoder-only transformer
Investigating grammatical abstraction in language models using few-shot learning of novel noun gender (2024.findings-eacl)
Copied to clipboard
| Challenge: | a new study shows that language models can generalise novel noun gender from one to two learning examples and apply it across agreement contexts. |
| Approach: | They conduct a noun learning experiment to assess whether a transformer and an LSTM can achieve human-like abstraction of grammatical gender in French. |
| Outcome: | The proposed models generalise gender from one to two learning examples and apply gender across agreement contexts, albeit with a bias for the masculine gender category. |
GraDeT-HTR: A Resource-Efficient Bengali Handwritten Text Recognition System utilizing Grapheme-based Tokenizer and Decoder-only Transformer (2025.emnlp-demos)
Copied to clipboard
| Challenge: | Bengali is the sixth most spoken language in the world, but handwritten text recognition systems for the language are underdeveloped. |
| Approach: | They propose a Bengali handwritten text recognition system that uses a decoder-only transformer to address the unique challenges of Bengali script. |
| Outcome: | The proposed system significantly improves on existing tokenizers on Bengali script. |
Prompt-Singer: Controllable Singing-Voice-Synthesis with Natural Language Prompt (2024.naacl-long)
Copied to clipboard
Yongqi Wang, Ruofan Hu, Rongjie Huang, Zhiqing Hong, Ruiqi Li, Wenrui Liu, Fuming You, Tao Jin, Zhou Zhao
| Challenge: | Recent singing-voice-synthesis methods lack ability to control style attributes of synthesized singing. |
| Approach: | They propose a singing-voice-synthesis method that enables attribute controlling on singer gender, vocal range and volume with natural language. |
| Outcome: | The proposed method achieves favorable control ability and audio quality. |
Probing Political Ideology in Large Language Models: How Latent Political Representations Generalize Across Tasks (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models encode rich internal representations of political ideology, but it remains unclear how these representations contribute to model decision-making. |
| Approach: | They apply inference-time interventions to steer a decoder-only transformer along learned ideological directions . they find that learned ideological representations generalize well to bias detection, but not as well to voting simulations . |
| Outcome: | The proposed model steers a transformer along learned ideological directions . political bias detection, voting preference simulation and bias neutralization are tested . |